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Abstract 

The objectives of this study were to develop a student self-assessment activity of a video-recorded counseling 
session and evaluate its impact on student self-perceptions of specific communication skills. This activity was 
incorporated into a core-communications course within the third professional year of a Doctor of Pharmacy 
curriculum. Student counseling sessions were video-recorded and released to students for self-assessment review. 
After watching their recorded counseling sessions, students completed an eight-question, paper-based survey 
which evaluated the impact of self-assessment on communication skill perceptions. Most students (95.6%) 
agreed or strongly agreed that their self-assessment process was valuable in developing their communication 
skills. The greatest change in assessment was associated with the use of eye contact (49.5%). Approximately 
40% of students noted changes in perceptions for appropriate rate of speech (42.1%), voice volume (39.8%), and 
facial expressions (37.2%). Overall, self-assessment of video-recorded counseling sessions impacted students’ 
perceptions of both verbal and non-verbal communication skills. 
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1. Introduction 

1.1 Background 

Self-assessment in professional education helps improve students’ knowledge, abilities, and values, and builds a 
foundation to support lifelong learning. As defined, self-assessment is a process that allows students to utilize 
established criteria to evaluate and reflect upon their learning to identify areas of improvement and competence 
(Motycka, Rose, Ried, & Brazeau, 2010). This competency is supported by the Accreditation Council of 
Pharmaceutical Education [ACPE] (2006) with recommendations to include activities that support 
self-assessment into the pharmacy curriculum. Stated under Guideline 11.1 regarding teaching methods, the 
standards call for the program to encourage and assist students to assume “responsibility for their own learning 
(including assessment of their learning needs, development of personal learning plans, self-assessment of the 
level of their knowledge skills, attitudes, and values, and their achievement of desired competencies and 
outcomes)” (ACPE, 2006). This responsibility also has been noted to be critical for their success after graduation, 
with self-appraisal being a component of continual professional development in pharmacy continuing education 
(Fjortoft, 2006). 

1.2 Review of Literature 

Many healthcare professions rely on self-assessment as a means of continuing professional development. 
Established research has demonstrated the limitations, however, of students in professional education to 
accurately self-assess their competence. In a systematic review of 20 physician self-assessments versus external 
observations, 13 showed little, no, or inverse associations between the two. These assessments included 
evaluation of clinical skills, teaching abilities, cultural competencies, and understanding of evidence based 
medicine (Davis et al., 2006). A study of self-assessment of communication skills in over 400 medical students 
found that student values differed significantly from the standardized patients for the majority of communication 
related skills (Gruppen et al., 1997). In a similar review of 80 senior students in a Doctor of Pharmacy program, 
students, on average, overestimated their communication skills and clinical knowledge, as compared to external 
raters (peers and expert) (Austin & Gregory, 2007). 
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Various studies have evaluated the use of video review in the self-assessment of clinical and/or communication 
skills (Del Mar & Isaacs, 1992; Hays, 1990; Hu, Tiemann, & Brunt, 2013; Nilsen & Baerheim, 2005; Ozcakar et 
al., 2009; Zick, Granieri, & Makoul, 2007). In addition, videotaping of student performance in the healthcare 
professions has been evaluated as a tool to improve the quality of self-assessments. This has been utilized for 
education in various healthcare professions including nursing, physical therapy, medicine, and surgery with 
varied results (Lane & Gottlieb, 2004; Palmer, Henry, & Rohe, 1985; Yoo, Son, Kim, & Park, 2009). In a study 
of a videotaped simulation by emergency medicine residents (n=17), residents rated their skills on a case before 
and after watching their videotaped performance. These assessments were then compared to those of a faculty 
evaluator. Video review did not improve the correlation of the resident scores to those of the faculty member 
(Sadosty et al., 2011). In a similar study of the use of video review for students (n=95) in basic life support 
training, the student assessments did not correlate with the expert evaluation either before or after video review 
(Vnuk, Owen, & Plummer, 2006). In a quantitative and qualitative review of the use of videotaping in simulated 
resuscitations and the impact of self-assessment for pediatric residents, feedback from study participants noted 
that the video review was beneficial, although challenging. Differences between observer assessments and 
self-assessments did not vary widely, however, the residents described less usefulness of observer assessments. 
After video review, the majority of residents lowered their scores, and the residents described this activity as the 
most useful aspect of the study. In a nursing course, self-assessment by video review of vital signs and 
communication skills lead to significantly better performance on an exam as compared to self-assessment by 
written description (Yoo et al., 2009). 

The impact of videotape review on pharmacy students’ self-assessments of communication skills was recently 
studied by Mort and Hansen (2010). First-year students in a pharmacy practice course participated in a digitally 
recorded counseling exercise. Correlation between students’ scores and faculty’s scores, pre- and post-videotape 
review were compared and found to be poor. Videotape review, however, was found to change students’ own 
rating of their counseling session. In the areas of technical information (details regarding indication, adverse 
effects, etc.) and interpersonal skills, the students’ pre- and post-video scores were significantly different 
(p—0.001 and pO.OOOl, respectively) than those prior to video review. Although the correlation to faculty 
ratings did not improve in this case, activities such as these may provide students the opportunity to practice and 
utilize the videotape as a tool to assist in the recognition of strengths and weaknesses (Mort & Hansen, 2010). 

Communication skills are an essential area for patient centered practice, as outlined in the principles of 
pharmaceutical care by the American Pharmacists Association [APhA] (1995). This skill is also supported by 
ACPE, as well as the recently revised Center for Advancement of Pharmacy Education (CAPE) outcomes 
(Medina et al., 2013). CAPE guidance suggests that entry level graduates should be able to communicate 
effectively both verbally and nonverbally, during interactions with patients, as well as with other individuals or 
groups. Areas for focus include technical information in addition to verbal and nonverbal skills, such as body 
position, eye contact, and gestures (Beardsley, Kimberlin, & Tindall, 2008). 

1.3 Course Description 

At The Ernest Mario School of Pharmacy, the Pharmacy Communications course series is comprised of two 
separate two-credit courses delivered over the fall and spring semesters of the third professional (P3) year of the 
Doctor of Pharmacy curriculum. Both courses utilize six small class size sections (approximately 30-35 students) 
and integrate lectures, group activities, and one-on-one counseling sessions for the development of student 
interpersonal, professional, and clinical communication skills. The weekly two hour course is coordinated by 
three full time faculty members with each section led and taught by two adjunct faculty members from 
post-graduate programs. Course materials, activities, and grading evaluations are developed by the faculty 
coordinators, with delivery of lecture materials and individual section direction by the adjunct faculty. 
Approximately one-half of the classes are comprised of one-on-one counseling sessions. Topics for counseling 
focus on a wide variety of prescription and nonprescription medications. 

Course topics are generally introduced via lectures delivered by the adjunct faculty, followed by an interactive 
breakout session to practice skills and/or apply material taught. Examples of topics discussed and evaluated 
include introduction to counseling, professionalism, how to give and receive feedback, presentation skills, 
journal clubs, and documentation of pharmaceutical care services. This course is well suited for the inclusion of 
a self-assessment activity due to small class sizes and inclusion of various individualized activities. 

Incorporation of self-assessment in the pharmacy curriculum is critical for students in order to utilize this skill 
upon graduation. Research has demonstrated varied results regarding the benefit of video review on 
self-assessment, with limited information in pharmacy education. In this study, the implementation of a 


27 




www.ccsenet.org/jel 


Journal of Education and Learning 


Vol. 3, No. 4; 2014 


video-recorded prescription medication counseling session and its impact on self-assessment of specific 
communication skills for students in the P3 year are described. 

2. Methods 

2.1 General Counseling Instruction 

This study was conducted over a three-week period of the Pharmacy Communications course involving one 
lecture/counseling practice day, one counseling practical day and one feedback/grade review day. The first class 
day introduced patient counseling through a small group lecture with sample counseling videos followed by an 
interactive breakout practice session. The introduction to counseling lecture provided a thorough review of the 
Indian Health Services (IHS) style counseling method which gauges a patient’s understanding of their 
medication(s) through the use of three open-ended questions regarding purpose, dosage/administration, and 
outcomes (Gardner, Boyce, & Herrier, 1991). Using the open-ended questions to guide the counseling session, 
the pharmacist supplements information provided by the patient and verifies that the patient understands the 
information. Verification occurs at the conclusion by having the patient describe how they will take the 
medication. Any discrepancies, missing information, or incorrect information can be addressed at the conclusion 
of the session (Gardner et al., 1991). 

The IHS counseling method review established the expected counseling style and structure for students to follow 
when counseling mock patients during practical assessments. Based on this method, a Pharmacy 
Communications I Counseling Guideline, as illustrated in Figure 1, was distributed to students in conjunction 
with the lecture material. Demonstration videos were used at the conclusion of the lecture to reinforce the major 
counseling components highlighted in the Counseling Guideline and covered during the lecture. 
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Pharmacy Communications I 
Counseling Guidelines 

Please keep in mind, this is a guide to assist you with counseling patients. This 
document is not all inclusive and should be used to help you organize your 
counseling session. You should tailor your counseling session according to your 
patient, medication(s), environment, and personal counseling style. 

1. Introduction 

a. Introduce/identify yourself 

b. Confirm identity of patient with a date of birth, address, etc 

c. Discuss purpose of counseling 

d. Confidentiality 

2. Medication 

^Identify the medication prescribed 

a. Open-Ended Question(s) 

i. What did your doctor tell you [medication name] is for? 

b. Potential Topics Discussed 

i. Name (brand/generic) 

ii. Strength 

iii. Dosage form 

iv. Appearance 

v. Indication or usage 

vi. Medication patient will be receiving (brand/generic & name) 

3. Dosing 

a. Open-Ended Question(s) 

i. How did your doctor tell you to take your [medication 
name]? 

b. Potential Topics Discussed 

i. Amount prescribed (* this can be discussed in Medication 
section but caution since it may lead you to prematurely 
discuss dosing) 

ii. Dose/amount to be taken 

iii. Route 

iv. Frequency 

v. Duration 

vi. Refills 

vii. Special directions (with respect to food, time of day, relation 
to other medications, etc) 

viii. Storage (*this can be incorporated as you see fit) 
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4. Safety/Side Effects/Expectations 

a. Open-Ended Question(s) 

i. What side effects did your doctor tell you to expect from 
[medication name]? 

b. Potential Topics Discussed 

i. Common side effects 

ii. Pertinent, less common side effects such as cough with ACE 
Inhibitors and what to do if they happen 

iii. What to do to help minimize adverse effects (e.g. stand up 
slowly with agents causing orthostatic hypotension) 

iv. Expectations for duration of side effects (will they be 
transient or continuous while on meds) 

v. Expectations for relief of symptoms or onset of medication 
(can also be incorporated into dosing section) 

vi. If applicable, what to do in response to AE or when to 
contact physician, RPh, ER (e.g. rash with antibiotic - stop 
medication, contact MD and RPh vs. anaphylactic reaction - 
seek urgent medical attention/ER and then inform MD and 
RPh) 

vii. Precautions/Warnings 

5. Recap 

a. Summary Sentence 

i. "I realize I went over a lot of information, so just to make 
sure I covered everything, please tell me how you are going 
to take your / Insert name of medication! ." - This is just one 
of many ways to recap your counseling session. 

Counseling Multiple Medications 

When counseling on multiple medications, it is recommended to completely 
counsel on individual medications (using the open ended questions). Attempting 
to counsel on all medications at one time may become confusing to the 
pharmacist and the patient. Use discretion based on content and comfort/style 
of delivering information. Overlapping issues, including drug interactions, may 
be handled towards the end of the counseling session or throughout (e.g. 
increase/decrease in electrolyte balance leading to a potentially clinically 
significant effect). Either way, it is important to identify and emphasize what the 
issue/drug interaction is along with any associated monitoring parameters, 
symptoms, follow-up, special counseling points, etc., the patient needs to know. 
Develop a style that you are comfortable with and that will effectively deliver the 
pertinent information to the patient. 


Figure 1. Pharmacy communications I counseling guidelines 


After the lecture and video review, students were provided the opportunity to practice their counseling skills in 
pairs. Each pair received two mock prescriptions (cephalexin and montelukast) and associated package inserts. 
Students were asked to individually prepare one prescription on which to counsel. Approximately thirty minutes 
were allotted to review the package insert and practice counseling. Faculty rotated throughout the room to 
observe counseling sessions and provide feedback to students. Once students felt comfortable with their 
counseling abilities, the class regrouped. Volunteers then demonstrated their counseling sessions in front of the 
class. Upon completion of each practice counseling session, feedback was solicited from the class to help 
develop students. Any students interested in practicing via this method could do so. 

2.2 Video-Recorded Counseling 

The next phase of the counseling series involved a counseling practical day scheduled for the following week. 
After the first week’s lecture/counseling practice day, a mock prescription (metoprolol succinate XL) and 
associated PI were posted on the university course management system, Sakai, for students to review in 
preparation for their counseling practical. The second class period (counseling practical day) consisted of a 
video-recorded counseling session and evaluators’ assessments of students’ counseling skills and clinical 
knowledge. Evaluators acted as patients and assessed students by utilizing a standardized grading form. The 
grading form incorporated the items listed in the Pharmacy Communications I Counseling Guideline. Grading 
forms also contained standardized responses for instructors based on the questions asked by students to maintain 
consistency among evaluators. Counseling practical sessions were limited to a maximum of ten minutes with a 
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goal for students to counsel within six minutes. Although additional counseling session practicals were 
conducted throughout the remainder of the semester, none were recorded. 

Upon completion of the practical session, counseling videos were made available for review. Students did not 
receive feedback or a grade at that time. Only individual students and faculty had access to their video. Students 
were instructed to review and critique their videos over the following week and prepare a list of strengths, areas 
needing improvement, and an action plan for self-improvement. This feedback was to be based off of their 
communication skills and the items listed in the Counseling Guideline. 

2.3 Study Survey and Feedback 

The final class period for the introduction to counseling portion of the course was the feedback/grade review day. 
Students completed an anonymous, eight-question, paper-based survey (Figure 2) prior to discussing their 
video-recorded counseling session. The survey evaluated the impact of the video review on their perceived 
communication skills and the activity’s value. Six questions addressed various aspects of communication, such 
as eye contact, facial expressions, body position, and quality of speech; one question addressed students’ opinion 
of the self-assessment activity in developing their communication skills. Additional comments regarding the 
self-assessment process were solicited through a final open-ended question. Convenience sampling was used to 
collect data. 


Directions: Please answer the following questions regarding non-verbal 

communication during your counseling session. PLEASE record (in pencil) all 

answers on the scantron provided. 

Please choose from the following list to describe how reviewing your video impacted the 
statements below (1.-6.) regarding your one-on-one counseling session. 

A. Better than I originally thought. 

B. No change from what I originally thought. 

C. Worse than I originally thought. 

1. I made direct eye contact with my patient for approximately 50-75% of the 
counseling session. 

2. My facial expressions were appropriate to display sincerity and willingness to 
help. 

3. My body position and gestures were open and warm. 

4. The rate of my speech was appropriate for understanding the information being 
conveyed. 

5. The volume of my voice allowed for clear communication, without disUirbing 
others or creating the opportunity of being overheard. 

6. The pitch of my voice was appropriate for the words used and varied to maintain 
interest. 

Please indicate how much you agree or disagree with the following statement regarding 
your video recorded one-on-one counseling session. 

A B C D E 

(strongly agree) (agree) (neutral) (disagree) (strongly disagree) 

7. I believe that self-assessment of my counseling video is valuable to develop my 
communication skills. 

8. Please add any additional comments regarding the self-assessment process below: 


Figure 2. Video self-evaluation survey 


Evaluation of the activity’s utility was measured by a change in student perception prior to and after counseling 
video review. Those who noted that video review led them to believe they did better than originally thought, 
were rated as initially underestimating their performance. An example of this would be a student who believed 
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that he/she had poor eye contact prior to reviewing the recorded counseling session and then rated his/her eye 
contact higher after reviewing the recorded counseling session video. Those who indicated that they believed 
they did worse than originally thought were noted as initially overestimating their performance. Any change 
from the initial perceived performance, regardless of overestimation or underestimation, was considered a 
reflection of the self-assessment providing a more accurate perception of the student’s performance. 

Once the survey was completed, students were matched up with their evaluators to discuss the self-assessment 
and receive feedback and a grade. Evaluators began the session by reviewing the strengths, areas needing 
improvement, and action plan for improvement. Students were also asked to describe how reviewing their video 
impacted their performance perception. Any remaining feedback in areas not covered during the student’s 
self-evaluation was then addressed by the evaluator. Students received full credit for the assignment regardless 
of their performance but were told their final raw score. Feedback regarding students’ performances, comments, 
and self-evaluations was obtained from evaluators at the conclusion of the review session. In addition, evaluators 
provided insight into the activity’s design, flow, and implementation for future semesters. 

2.4 Ethics and Procedures 

A notice of exemption from IRB review was granted by the Office of Research and Sponsored Programs at 
Rutgers, The State University of New Jersey. 

3. Results 

3.1 Participants ’ Characteristics 

The Pharmacy Communications I course included 198 P3 students. Approximately 95% (n=188) of students 
completed the self-assessment survey. Although demographic data were not collected with the in-class survey, 
Table 1 highlights the Rutgers P3 class data from the American Association of Colleges of Pharmacy (AACP) 
2011 Enrollment Survey. 


Table 1. AACP 2011 enrollment survey data (n=197) 



White 

Black/ 

African 

American 

Hispanic/ 

Latino 

Asian 

>2 

Races 

Race/Ethnicity 

Unknown 

Foreign/ 

International 

Total 

Male 

9.1% 

2.0% 

1.0% 

25.4% 

0% 

2.5% 

2.0% 

42.1% 

Female 

18.3% 

3.1% 

3.1% 

31.5% 

1.0% 

0% 

1.0% 

57.9% 

Total 

27.4% 

5.1% 

4.1% 

56.9% 

1.0% 

2.5% 

3.0% 



3.2 Survey Data 

Figure 1 illustrates the impact of self-assessment on the perceptions of student communication skills. 
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Communication Skill 
■ Underestimated * No Change ■ Overestimated 


Figure 3. Impact of video self-assessment on initial perceptions of student communication skills (n=188) 


Overall, 95.6% (173/181) of students agreed (15.5%) or strongly agreed (80.1%) that the self-assessment process 
was valuable in the development of their communication skills. Only one student (0.6%) disagreed and 3.9% of 
students were neutral regarding the activity’s value. Eye contact was identified as the skill with the greatest 
difference in assessment perception after reviewing the recorded counseling session in approximately 50% of 
students; 35.1% underestimated and 14.4% overestimated this ability. Approximately 40% of students noted 
changes in perceptions regarding appropriate rate of speech (42.1%), voice volume (39.8%), and facial 
expressions (37.2%). Body positioning/gestures and voice pitch were associated with the fewest changes in 
perception, at approximately 33%. No comments were provided for the final open-ended question regarding 
additional feedback pertaining to the self-assessment process. 

4. Discussion 

In this study, a counseling activity with video review and self-assessment components capable of assisting 
students in identifying changes in communication skills perceptions was successfully implemented in a 
Pharmacy Communications class. Consistent with previous studies (Del Mar & Isaacs, 1992; Nilsen & Baerheim, 
2005; Yoo et al., 2009), a majority of students agreed that the self-assessment exercise was beneficial in the 
development of their communication skills. 

Similar to findings by Mort and Hansen (2010), a large percentage of students in our study changed their score 
after reviewing their video-recorded counseling sessions. In Mort and Hansen’s (2010) assessment, pre-video 
self-assessments were scored significantly (p<0.0001) lower than the post-video assessments. Our study found 
that approximately 39% of students had a perceived change in performance after watching their videos with 27% 
and 13% indicating they underestimated or overestimated their nonverbal communication skills, respectively. 
These changes are reflective of research by Zick et al. (2007) which indicates that video review is effective at 
providing valuable insight into students’ strengths and weaknesses. Their findings illustrate that approximately 
18% of students are more aware of their strengths and weaknesses by observing the impact of their interactions 
with simulated patients (Zick et al., 2007). 

Results from our study indicate a varying degree of change in the verbal and nonverbal communication skills 
assessed. The greatest difference in post-video review assessment was associated with the extent of eye contact 
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during the counseling session. Since this form of non-verbal communication may have been difficult to assess 
without self-observation, viewing the video could have better illustrated the student’s ability to make eye contact. 
Underestimations of eye contact may have been related to perceived nervousness or referring to the prescription 
blank for counseling information. Several studies have described the varying degrees of anxiety that are 
associated with the anticipation of being video-recorded during simulated patient interactions (Del Mar & Isaacs, 
1992; Lindon-Morris & Laidlaw, 2014; Nilsen & Baerheim, 2005). Del Mar and Isaacs (1992) reported that 
approximately 25% of students whose consultation skills were video-recorded felt intimidated by the camera. 
The simple presence of a video camera may have impacted students’ abilities to accurately recall and evaluate 
their counseling performance prior to watching their video. 

The category of body position/gesture produced the least amount of change in self-assessment evaluations of 
nonverbal communication skills. Variability in this category may have been reduced due to room design and 
space limitations. Since the counseling rooms are small with an immovable table between the student and 
evaluator, students were directed to sit in a specific chair and location within the room. This predetermined 
seating arrangement and limited space for repositioning predisposed students to sit in an appropriate manner. In 
addition, variations in hand gestures may have been reduced since students were provided the opportunity to use 
notes or the paper prescription handout available during the session. This may have occupied their hands and 
minimized distractive movement. 

It has been often found that students have a high grade orientation, described as a sole focus on the obtainment of 
grades as a function of learning (Janzow & Eison, 1990). This may become a barrier when discussing and 
receiving constructive feedback. Using self-assessment and video-recorded reviews of activities may help 
facilitate discussions and shift the attention away from grades and towards improving students’ communication 
and clinical skills. Many faculty evaluators mentioned that beginning the feedback sessions with the student 
self-assessments appeared to improve constructive dialogue. Evaluators also indicated it allowed them to focus 
on how communication skills could be improved while avoiding a grade-driven discussion. 

4.1 Limitations 

Some limitations identified throughout the self-assessment activity may impact its use in practice. As reported by 
Ozcakar et al. (2009), more time is required for feedback associated with videotaping compared to only verbal 
feedback. In addition, resources including video equipment, a course management system, and use of class time 
were all important factors considered when implementing this activity. Four hours of class time over two weeks 
were dedicated to recording and uploading videos, completing evaluations, and providing students feedback. 
Students also spent time reviewing their counseling sessions and preparing self-evaluations outside of the 
classroom. 

Measures were not implemented to ensure that students reviewed their videos prior to developing their 
self-assessments and completing the survey. Students may have completed the self-evaluation and surveys 
without watching their counseling videos. Based on class size and available resources, study activities such as 
video reviews, self-evaluations, survey completion, and feedback discussions could not be performed 
immediately after the counseling session. As a result, the self-assessment survey was completed the following 
week and assessed students’ perceptions of change based on memory rather than a direct comparison of two 
separate self-assessment evaluations (pre- and post-) performed in proximity to the counseling session. Adding a 
pre-video assessment could establish a baseline to which post-video assessments could be compared. This could 
also help evaluators provide feedback immediately following the evaluation rather than taking notes for future 
discussions. 

Unlike previous studies involving video-recorded activities (Mort & Hansen, 2010; Ozcakar et al., 2009), this 
evaluation did not directly compare students’ self-assessments to evaluators’ assessments. Subsequently, 
longitudinal assessments throughout the semester were not performed. Since the activity was conducted only 
once, at the beginning of the semester, its impact on student development over the semester and comparison at 
the end of semester could not be determined. Including grade comparisons within the activity and throughout the 
semester to determine if students identified similar communication skill strengths and weaknesses could be 
incorporated into future studies. In addition, faculty evaluators were not directly surveyed for their feedback on 
the self-assessment activity and student performance but anecdotal comments were obtained. 

Finally, student demographic data were not collected or assessed with respect to responses provided. Since 95% 
of the class completed the survey, data from the AACP 2011 Enrollment survey for the Rutgers P3 students was 
included as an overall overview of student characteristics. Correlations among student demographics, responses, 
and changes in perceptions could not be made. 
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5. Conclusion 

Self-assessment of video-recorded counseling sessions impacted students’ perceptions of their verbal and 
non-verbal communication skills. Student scores changed after video review, with the most notable change 
occurring in the category of eye contact. Overall, the majority of students found the video recording to be a 
beneficial tool in their development. Incorporating self-assessment activities, such as a counseling video review, 
may assist in identifying more accurate perceptions of communication skills while focusing more attention on 
student development and performance. A video-recorded counseling session may also help improve students’ 
awareness of isolated communication skills such as eye contact, speech rate, voice volume, and facial 
expressions. 
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